Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

packager lambda for QPE #4304

Merged
merged 38 commits into from
Feb 21, 2025
Merged

packager lambda for QPE #4304

merged 38 commits into from
Feb 21, 2025

Conversation

sir-sigurd
Copy link
Member

@sir-sigurd sir-sigurd commented Jan 29, 2025

Description

TODO

  • Unit tests
  • Changelog entry (skip if change is not significant to end users, e.g. docs only)
  • revert deploy
  • change default prefix/suffix

Sorry, something went wrong.

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 39.07%. Comparing base (432b951) to head (dbe757c).
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4304      +/-   ##
==========================================
- Coverage   39.09%   39.07%   -0.03%     
==========================================
  Files         783      787       +4     
  Lines       34756    34813      +57     
  Branches     5522     5525       +3     
==========================================
+ Hits        13589    13604      +15     
- Misses      19985    20026      +41     
- Partials     1182     1183       +1     
Flag Coverage Δ
api-python 91.39% <ø> (ø)
catalog 18.09% <ø> (-0.04%) ⬇️
lambda 91.53% <100.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@drernie
Copy link
Member

drernie commented Feb 19, 2025

The inference is reasonable. I naively assume _ always means "/". Could we use different chars ("-") for other invalid chars, or is that added complexity?

@sir-sigurd
Copy link
Member Author

@greptileai
review, pls

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds a new package management functionality to the pkgpush Lambda function, focusing on automated package creation from S3 prefixes.

  • Added PackagerEvent class in t4_lambda_pkgpush/__init__.py with concerning use of global variables for boto sessions and multiple assertions instead of proper error handling
  • Added rfc3986==2.0.0 dependency for URI parsing in both requirements.txt and setup.py
  • Modified .github/workflows/deploy-lambdas.yaml to only deploy pkgpush Lambda, but deployment trigger configuration needs review
  • Added basic test coverage in test_packager.py but lacks error scenarios and edge cases
  • SQS handler explicitly refuses to process multiple records which may impact performance

5 file(s) reviewed, 6 comment(s)
Edit PR Review Bot Settings | Greptile

sir-sigurd and others added 3 commits February 20, 2025 22:07
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@sir-sigurd sir-sigurd marked this pull request as ready for review February 20, 2025 20:00
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

This PR updates the default package naming convention in the pkgpush Lambda function, changing from 'quilt-packager/pkg' to 'package/null' with corresponding test updates.

  • Modified default package name constants in /lambdas/pkgpush/tests/test_packager.py to use DEFAULT_PKG_NAME_PREFIX = "package" and DEFAULT_PKG_NAME_SUFFIX = "null"
  • Added comprehensive test cases in test_packager.py to validate package name inference with the new defaults
  • Updated package name validation logic in t4_lambda_pkgpush/__init__.py to handle the new naming scheme
  • Added test coverage for edge cases like empty prefixes and special characters in package names

2 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile

@sir-sigurd sir-sigurd requested a review from nl0 February 20, 2025 20:05
nl0
nl0 previously approved these changes Feb 21, 2025
Copy link
Member

@nl0 nl0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks sane

@sir-sigurd sir-sigurd requested a review from nl0 February 21, 2025 09:06
@sir-sigurd sir-sigurd changed the title WIP: packager packager lambda for QPE Feb 21, 2025
@sir-sigurd sir-sigurd added this pull request to the merge queue Feb 21, 2025
Merged via the queue into master with commit 07c85ce Feb 21, 2025
37 checks passed
@sir-sigurd sir-sigurd deleted the packager branch February 21, 2025 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants